Method and program for extracting silhouette image and method and program for constructing three dimensional model

ABSTRACT

A present invention provides a method and a program for extracting the high accuracy silhouette by relatively simple process not using manual labor or special photography environment. A method for extracting the high accuracy silhouette comprises: extracting a number of first silhouettes from a number of object images and a number of background images by a background subtraction; constructing first visual hull from a number of the first silhouettes by a shape from silhouette method; constructing second visual hull by process to repair missed parts and/or to remove unwanted regions in first visual hull; and extracting a number of second silhouettes from the second visual hull.

PRIORITY CLAIM

This application claims priority from Japanese patent application No.2008-331762, filed on Dec. 26, 2008, which is incorporated herein byreference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a method and a program for extracting asilhouette image, and a method and a program for constructing 3D voxelmodel (visual hull). More specifically, the present invention relates toa background subtraction method and a program for extracting thesilhouette from an image which an object is captured and an image whichonly a background is captured, and a 3D modeling method and a programfor constructing the visual hull from a number of the silhouettesextracted by the above by applying a shape from silhouette method.

2. Description of the Related Art

Heretofore, a background subtraction which extracts the silhouetterepresenting the object from the image which the object is captured anda 3D modeling which constructs visual hull from a number of the makeimages by applying the shape from silhouette method are performedseparately. For this reason, in order to construct high accuracy visualhull, it was necessary to extract a high accuracy silhouette and specialenvironment such as a blue back was necessary. Japanese patentpublication No. 2007-17364 and Toyoura et al “Silhouette Refinement forVisual Hull with Random Pattern Background”, the 2005 IEICE GeneralConference, D-12-133 describe a method for improving accuracy of thesilhouette, the method repairs a missed part of the silhouetteexercising ingenuity in a background subtraction and using colorinformation of the visual hull.

BRIEF SUMMARY OF THE INVENTION

A conventional method needed firstly a sufficiently high accuracysilhouette in order to construct high accuracy visual hull. Therefore,there was a problem to have to extract the high accuracy silhouette withcomplicatedly calculating and using manual labor or special photographyenvironment such as the blue back.

Therefore, it is an object of a present invention to provide a methodand a program for extracting the high accuracy silhouette by relativelysimple process not using manual labor or special photographyenvironment, and a method and a program for constructing visual hullfrom said silhouette.

To realize the above object, according to a method for extracting anumber of silhouettes of the present invention, a method for extractinga number of silhouettes representing the object from a number of objectimages which an object and a background are captured and a number ofbackground images which only a background is captured comprises: a firstextract step of extracting a number of first silhouettes from a numberof said object images and a number of said background images by abackground subtraction; a first construct step of constructing firstvisual hull from a number of said first silhouettes by a shape fromsilhouette method; a second construct step of constructing second visualhull by process to repair missed parts and/or to remove unwanted regionsin first visual hull; and a second extract step of extracting a numberof second silhouettes from said second visual hull.

Further, it is also preferable that said second extract step from saidfirst construct step are repeated with assuming a number of said secondsilhouettes to be a number of said first silhouettes at said firstconstruct step.

Further, it is also preferable that said second construct stepcomprises: a sub step of acquiring a number of first slice images ofsaid first visual hull from an x-axis, a y-axis and a z-axis direction;and a sub step of performing a filter process to a number of said firstslice images and constructing second visual hull according to a resultof said filter process.

Further, it is also preferable that said sub step of constructing secondvisual hull is a step of constructing said second visual hull byperforming a filter process to a number of said first slice images, bysearching pixels became white by the filter process, and by repairingthree dimensional coordinates of said first visual hull corresponding tosaid pixels.

Further, it is also preferable that said second extract step comprises:a sub step of performing a filter process to a number of said firstslice images, searching pixels became white by the filter process,projective-transforming said pixels into the silhouettes, and extractinga number of third silhouettes; a sub step of acquiring a number ofsecond slice images of said second visual hull from the y-axisdirection; a sub step of extracting a number of fourth silhouettes, bysearching white pixels of a number of said second slice images, and byprojective-transforming said pixels into the silhouettes; and a sub stepof extracting a number of second silhouettes by performing a filterprocess to a number of said fourth silhouettes, by making white ofpixels that both of a number of said fourth silhouettes performed thefilter process and a number of said third silhouettes are white, and bymaking black of other pixels.

Further, it is also preferable that said method for extracting a numberof silhouettes further comprises a step of performing a filter processto a number of said second silhouettes after said second extract step.

To realize the above object, according to a program for extracting anumber of silhouettes of the present invention, a program for extractinga number of silhouettes representing the object from a number of objectimages which an object and a background are captured and a number ofbackground images which only a background is captured comprises: a firstextract means for extracting a number of first silhouettes from a numberof said object images and a number of said background images by abackground subtraction; a first construct means for constructing firstvisual hull from a number of said first silhouettes by a shape fromsilhouette method; a second construct means for constructing secondvisual hull by process to repair missed parts and/or to remove unwantedregions in first visual hull; and a second extract means for extractinga number of second silhouettes from said second visual hull.

To realize the above object, according to a method for constructingvisual hull of the present invention, a method for constructing visualhull from a number of object images which an object and a background arecaptured and a number of background images which only a background iscaptured comprises: a first extract step of extracting a number of firstsilhouettes from a number of said object images and a number of saidbackground images by a background subtraction; a first construct step ofconstructing first visual hull from a number of said first silhouettesby a shape from silhouette method; a second construct step ofconstructing second visual hull by process to repair missed parts and/orto remove unwanted regions in first visual hull; a second extract stepof extracting a number of second silhouettes from said second visualhull; and a third construct step of constructing third visual hull froma number of said second silhouettes by the shape from silhouette method.

To realize the above object, according to a program for constructingvisual hull of the present invention, a program for constructing visualhull from a number of object images which an object and a background arecaptured and a number of background images which only a background iscaptured comprises: a first extract means for extracting a number offirst silhouettes from a number of said object images and a number ofsaid background images by a background subtraction; a first constructmeans for constructing first visual hull from a number of said firstsilhouettes by a shape from silhouette method; a second construct meansfor constructing second visual hull by process to repair missed partsand/or to remove unwanted regions in first visual hull; a second extractmeans for extracting a number of second silhouettes from said secondvisual hull; and a third construct means for constructing third visualhull from a number of said second silhouettes by the shape fromsilhouette method.

According to the present invention, the method and the program forextracting the silhouettes perform the improvement of the accuracy ofthe silhouette and the visual hull with interpolation by reflecting theinformation of visual hull to the silhouette. The method and the programdoes not need firstly the sufficiently high accuracy silhouette, and canextract the high accuracy silhouette without complicatedly calculating,and can construct the high accuracy visual hull from the high accuracysilhouette. Further, the present invention does not rely on the specialphotography environment and can apply general images.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

FIG. 1 shows a flow chart showing a method for extracting the silhouetteand a method for constructing visual hull according to the presentinvention;

FIG. 2 shows an example of a silhouette obtained from the camera image;

FIGS. 3 a to 3 c show an example of visual hull constructed from thesilhouette of FIG. 2;

FIGS. 4 a to 4 c show an example of visual hull that missed parts of thevisual hull of FIG. 3 are repaired;

FIG. 5 shows a silhouette that missed parts of the silhouette of FIG. 2are repaired;

FIG. 6 shows a silhouette that unnecessary parts of the silhouette ofFIG. 5 are removed;

FIG. 7 shows a silhouette that a filter process is performed to thesilhouette of FIG. 6;

FIG. 8 shows a silhouette extracted by performing the step 9 from thestep 3 of the present invention three times;

FIG. 9 shows a silhouette extracted by performing the step 9 from thestep 3 of the present invention five times;

FIG. 10 shows a silhouette extracted by performing the step 9 from thestep 3 of the present invention seven times;

FIG. 11 shows a silhouette extracted by performing the step 9 from thestep 3 of the present invention nine times;

FIGS. 12 a to 12 c show visual hull constructed from the silhouette ofFIG. 7;

FIGS. 13 a to 13 c show visual hull constructed from the silhouette ofFIG. 8;

FIGS. 14 a to 14 c show visual hull constructed from the silhouette ofFIG. 9;

FIGS. 15 a to 15 c show visual hull constructed from the silhouette ofFIG. 10; and

FIGS. 16 a to 16 c show visual hull constructed from the silhouette ofFIG. 11.

DETAILED DESCRIPTION OF THE INVENTION

An embodiment of a method and a program for extracting a silhouette anda method and a program for constructing visual hull according to thepresent invention will be described below with reference to thedrawings. FIG. 1 shows a flow chart showing a method for extracting thesilhouette and a method for constructing visual hull according to thepresent invention. The embodiment will be described below with referenceto the flow chart.

Step 1: A number of object images and background images placed in acircle are obtained. A number of calibrated cameras are placed in thecircle, and the object images including the object and the backgroundand the background images only including the background are capturedwith said cameras, and then the object images and the background imagescaptured from a number of different directions are obtained. Forexample, when 30 cameras are placed, the object images and thebackground images are obtained 30 pieces, respectively.

Step 2: A number of silhouettes are extracted from said object imagesand said background images by a background subtraction. Since thesesilhouettes are obtained by the simple conventional backgroundsubtraction, the accuracy of these silhouettes is not high. Thesilhouettes are extracted for the number of the camera. For example,when 30 cameras are placed, 30 pieces of silhouettes are extracted.

Step 3: Visual hull are construct by applying a shape from silhouettemethod. Since the accuracy of the visual hull depends on the accuracy ofthe silhouettes, when the silhouettes extracted by the step 2 are used,the accuracy of the constructed visual hull is not high.

Step 4: Slice images of the above visual hull are acquired. Consideringthat visual hull is the collection of the slice images from a certaindirection, the slice images of the visual hull are acquired from anx-axis, a y-axis and a z-axis direction. The slice images get the numberof sheets of the coordinate range with each axis. For example, when they-axis coordinate range of the visual hull is −256 −255, 512 pieces ofslice images are acquired. Note that, the y-axis is vertical direction,and the x-axis and the z-axis are horizontal directions, respectively.

Step 5: The visual hull in which missed parts are repaired areconstructed. Since there may be a case that the slice images of step 4are acquired from the visual hull which is not high accuracy, they mayinclude missed parts in which a white point representing the objectregion is misclassified as black representing background or unwantedregions in which white is misclassified as black. Therefore, a filterprocess is performed to the acquired slice images from each direction(the x-axis, the y-axis, the z-axis). For example, some of missed partsare repaired by applying a gauss filter, and the unwanted regions in thevisual hull are removed by applying a median filter. In this way, thefiltered slice images are obtained. Next, the filtered slice images arecompared with the no filtered slice images, and pixels that became thewhite newly (i.e. the pixels that were repaired by the filter process)are searched, and then the three dimensional coordinates of the visualhull corresponding to these pixels are repaired. For example, when thepixel that became the white is y-coordinate y1, z-coordinate z1 in theslice image acquired in coordinate x1 of the x-axis, the threedimensional coordinate (x1, y1, z1) of visual hull is repaired. Theabove-mentioned process is performed to all slice images, and the visualhull that the missed parts are repaired is acquired.

Step 6: The silhouettes in which a number of missed parts are repairedare extracted. The three dimensional coordinates of the step 5 areprojective-transformed into each silhouette, and the correspondingpixels in each silhouette become white. In other words, the threedimensional coordinate of the slice image is projective-transformed, andan image watched from the position where each silhouette is captured ismade, and the pixels corresponding to the three dimensional coordinateof the step 5 in the said image become white. Thereby, the silhouettesin which the missed parts are repaired are extracted.

Step 7: Slice images of the visual hull acquired by the above step 5 areacquired from the y-axis direction. The slice images get the number ofsheets of the coordinate range with each axis as well as the step 4.

Step 8: A number of silhouettes in which unnecessary parts are removedare extracted. The three dimensional coordinates corresponding the whitepixels of the slice images from the y-axis direction acquired by thestep 7 are projective-transformed into the each silhouette, and thecorresponding pixels in each silhouette become white. A filter processis performed to the silhouettes obtained above. The silhouettes obtainedabove are compared with the silhouettes extracted by the step 6, pixelsthat both images are white make white, and other pixels make black. Theabove-mentioned process is performed to all silhouettes, and thesilhouettes in which the unnecessary parts are removed are extracted.

Step 9: The silhouettes in which a number of missed parts are repairedare extracted. A filter process is performed to the silhouettes obtainedabove, so that the missed parts are further repaired. Thus, the missedparts are repaired (step 6), and the unnecessary parts are deleted (step8), and the missed parts are repaired (step 9), the higher accuracysilhouettes are extracted by these process.

Step 10: If the accuracy of the silhouettes are enough, high accuracyvisual hull are constructed by the shape from silhouette method fromthese silhouettes. If the accuracy of the silhouettes are not enough,with assuming the silhouettes obtained by the step 9 to be an input ofthe step 3, the step 9 from the step 3 are repeated, the accuracy of thesilhouettes and the visual hull are gradually updated.

Generally, when the shape from silhouette method is used, theunnecessary parts such as shadows in the silhouettes do not affect thegeneration of the visual hull too much. However, the missed parts in aperson mask affect the generation of the visual hull too much. It isnecessary to repair the missed parts in the person mask.

Also, the pixels that there is a person in the slice image of visualhull (i.e., the pixels of the white rather than the black) surely becomethe white in the corresponding pixels of each silhouette. However, thepixels that there is not a target object (the black pixels) may not benecessarily black in each silhouette.

Therefore, the pixels of the white in the slice images are important,and the improvement of the accuracy of the slice images will repair inthe missed parts little by little. The accuracy of the slice imagesimproves, and the missed parts of the visual hull are repaired little bylittle by repeating the step 9 from the step 3.

Next, the improvement of the silhouettes is shown by real images. FIG. 2shows an example of a silhouette obtained from the camera image. FIG. 2is a silhouette generated by the step 2 by the simple backgroundsubtraction, and it is found that the accuracy is not high.

FIGS. 3 a to 3 c show an example of visual hull constructed from thesilhouette of FIG. 2. Since the visual hull are three dimensions, animage (a) watched from the horizontal direction (the x-axis direction),an image (b) watched from the vertical direction (the y-axis direction),and an image (c) watched from the horizontal direction (the z-axisdirection) are shown. Note that, FIG. 3 b is reduced 50% in comparisonwith FIG. 3 a and FIG. 3 c. Since these visual hull are constructed bythe silhouette of FIG. 2 which is not high accuracy, the accuracy is nothigh. The pixels which should be white are not repaired and are black,and originally black pixels are white.

FIGS. 4 a to 4 c show an example of visual hull that missed parts of thevisual hull of FIG. 3 are repaired. It is the visual hull after havingrepaired the missed parts of the visual hull of FIG. 3 by the step 5.These figures also show an image (a) watched from the x-axis direction,an image (b) watched from the y-axis direction, and an image (c) watchedfrom the z-axis direction, and FIG. 4 b is reduced 50% in comparisonwith FIG. 4 a and FIG. 4 c. When FIG. 4 compares with FIG. 3, forexample, it is found that missed parts in the second person from theleft of the x-axis direction are repaired.

FIG. 5 shows a silhouette that missed parts of the silhouette of FIG. 2are repaired. It is the silhouette that the process of the step 5 andthe step 6 are performed. When FIG. 5 compares with FIG. 2, missed partsin the part of the person are repaired. In other words, it is found thata black pixel becomes the white.

FIG. 6 shows a silhouette that unnecessary parts of the silhouette ofFIG. 5 are removed. It is the silhouette that the process of the step 8is performed. When FIG. 6 compares with FIG. 5, it is found that theunnecessary parts are removed. Note that, since the silhouette of FIG. 6extracts white parts with both of the slice image extracted from they-axis direction of the step 7 and the silhouette of FIG. 5, backgroundparts of the silhouette of FIG. 6 are removed.

FIG. 7 shows a silhouette that a filter process is performed to thesilhouette of FIG. 6. It is the silhouette that the process of the step9 is performed. When FIG. 7 compares with FIG. 6, it is found thatmissed parts are further repaired by the filter process.

Note that, the silhouettes of FIG. 2, FIG. 5, FIG. 6, and FIG. 7 are onepiece of a number of silhouettes, and the process was performed to thesilhouettes for the number of the cameras.

When the silhouette of FIG. 2 compares with the silhouette of FIG. 7, itis found that the accuracy of silhouettes largely improves by performingthe step 9 from the step 3 of the present invention once.

The accuracy of silhouettes can further improve by performing the step 9from the step 3 of the present invention several times. FIG. 8 shows asilhouette extracted by performing the step 9 from the step 3 of thepresent invention three times. FIG. 9 shows a silhouette extracted byperforming the step 9 from the step 3 of the present invention fivetimes. FIG. 10 shows a silhouette extracted by performing the step 9from the step 3 of the present invention seven times. FIG. 11 shows asilhouette extracted by performing the step 9 from the step 3 of thepresent invention nine times. When the silhouettes of FIG. 7, FIG. 8,FIG. 9, FIG. 10, and FIG. 11 compare with each other, it can be realizedthat the accuracy of the silhouettes little by little improves. Forexample, a point pointed out with an arrow 1 is originally a point whichshould be white in the people. When FIG. 8 compares with FIG. 7, it isfound that a missed part becomes small at FIG. 8, and it is found thatthe missed part is completely repaired at FIG. 9, FIG. 10, and FIG. 11.Also, a point pointed out with an arrow 2 is originally a backgroundpoint which should be black. When FIG. 7, FIG. 8, and FIG. 9 comparewith each other, it is found that a point of the white becomes small,and it is found that the point of the white completely disappears atFIG. 10 and FIG. 11.

Finally, the visual hulls constructed from silhouettes of FIG. 7, FIG.8, FIG. 9, FIG. 10, and FIG. 11 are shown. FIGS. 12 a to 12 c showvisual hull constructed from the silhouette of FIG. 7, FIGS. 13 a to 13c show visual hull constructed from the silhouette of FIG. 8, FIGS. 14 ato 14 c show visual hull constructed from the silhouette of FIG. 9,FIGS. 15 a to 15 c show visual hull constructed from the silhouette ofFIG. 10, and FIGS. 16 a to 16 c show visual hull constructed from thesilhouette of FIG. 11, respectively. These figures show an image (a)watched from the x-axis direction, an image (b) watched from the y-axisdirection, and an image (c) watched from the z-axis direction, and theimage (b) is reduced 50% in comparison with the image (a) and the image(c). It is found that the accuracy of the visual hull improves withimprovement of the accuracy of the silhouettes, respectively. Forexample, in the second person from the left of the x-axis direction, itis found that a missed part gradually shrinks as it advances with FIG.12, FIG. 13, and FIG. 14 and it is found that the missed part iscompletely repaired at FIG. 15 and FIG. 16. Also, it is found that apart below the foot of second person from the right (it is a backgroundand should be black) gradually shrinks as it advances with FIG. 12, FIG.13, FIG. 14, and FIG. 15, and it is found that it completely disappearsat FIG. 16. In this way, it is found that the visual hull improved theaccuracy are constructed from the silhouette improved the accuracy.

All the foregoing embodiments are by way of example of the presentinvention only and not intended to be limiting, and many widelydifferent alternations and modifications of the present invention may beconstructed without departing from the spirit and scope of the presentinvention. Accordingly, the present invention is limited only as definedin the following claims and equivalents thereto.

1. A method for extracting a number of silhouettes representing anobject from a number of object images in which the object and abackground are captured and a number of background images in which onlythe background is captured comprising: a first extract step ofextracting a number of first silhouettes from the number of said objectimages and the number of said background images by a backgroundsubtraction; a first construct step of constructing a first visual hullfrom the number of said first silhouettes by a shape from silhouettemethod; a second construct step of constructing a second visual hull bya process to repair missed parts or to remove unwanted regions in saidfirst visual hull; and a second extract step of extracting a number ofsecond silhouettes from said second visual hull, wherein the steps areperformed by a processor, said second construct step further including asub step of acquiring a number of first slice images of said firstvisual hull from an x-axis, a y-axis and a z-axis direction; and a substep of performing a filter process to the number of said first sliceimages and constructing said second visual hull according to a result ofsaid filter process, and said sub step of constructing said secondvisual hull is a step of constructing said second visual hull byperforming a filter process to the number of said first slice images, bysearching for pixels that became white by the filter process, whereinwhite represents the object region, and black represents background, andby repairing three dimensional coordinates of said first visual hullcorresponding to said pixels.
 2. The method for extracting the number ofsilhouettes according to claim 1, wherein said second extract step fromsaid first construct step are repeated with assuming the number of saidsecond silhouettes to be the number of said first silhouettes at saidfirst construct step.
 3. The method for extracting the number ofsilhouettes according to claim 1, wherein said second extract stepfurther comprises: a sub step of projective-transforming said pixelsinto the silhouettes, and extracting a number of third silhouettes; asub step of acquiring a number of second slice images of said secondvisual hull from the y-axis direction; a sub step of extracting a numberof fourth silhouettes, by searching white pixels of the number of saidsecond slice images, and by projective-transforming said pixels into thesilhouettes; and a sub step of extracting a number of second silhouettesby performing a filter process to the number of said fourth silhouettes,by making white of pixels that both of the number of said fourthsilhouettes performed the filter process and the number of said thirdsilhouettes are white, and by making black of other pixels.
 4. Themethod for extracting the number of silhouettes according to claim 1,wherein said method for extracting the number of silhouettes furthercomprises a step of performing a filter process to the number of saidsecond silhouettes after said second extract step.
 5. A non-transitorycomputer readable medium storing a program for extracting a number ofsilhouettes representing an object from a number of object images inwhich the object and a background are captured and a number ofbackground images in which only the background is captured, the program,when executed by a computer, causing the computer to perform a methodcomprising: extracting a number of first silhouettes from the number ofsaid object images and the number of said background images by abackground subtraction; constructing a first visual hull from the numberof said first silhouettes by a shape from silhouette method;constructing a second visual hull by a process to repair missed parts orto remove unwanted regions in said first visual hull; and extracting anumber of second silhouettes from said second visual hull, wherein saidsecond construct means includes a sub means for acquiring a number offirst slice images of said first visual hull from an x-axis, a y-axisand a z-axis direction; and a sub means for performing a filter processto the number of said first slice images and constructing said secondvisual hull according to a result of said filter process, and said submeans for constructing said second visual hull is a means forconstructing said second visual hull by performing a filter process tothe number of said first slice images, by searching for pixels thatbecame white by the filter process, wherein white represents the objectregion, and black represents background, and by repairing threedimensional coordinates of said first visual hull corresponding to saidpixels.
 6. A method for constructing a visual hull from a number ofobject images in which an object and a background are captured and anumber of background images in which only the background is capturedcomprising: a first extract step of extracting a number of firstsilhouettes from the number of said object images and the number of saidbackground images by a background subtraction; a first construct step ofconstructing a first visual hull from the number of said firstsilhouettes by a shape from silhouette method; a second construct stepof constructing a second visual hull by a process to repair missed partsor to remove unwanted regions in said first visual hull; a secondextract step of extracting a number of second silhouettes from saidsecond visual hull; and a third construct step of constructing a thirdvisual hull from the number of said second silhouettes by the shape fromsilhouette method, wherein the steps are performed by a processor, saidsecond construct step includes a sub step of acquiring a number of firstslice images of said first visual hull from an x-axis, a y-axis and az-axis direction; and a sub step of performing a filter process to thenumber of said first slice images and constructing said second visualhull according to a result of said filter process, and said sub step ofconstructing said second visual hull is a step of constructing saidsecond visual hull by performing a filter process to the number of saidfirst slice images, by searching for pixels that became white by thefilter process, wherein white represents the object region, and blackrepresents background, and by repairing three dimensional coordinates ofsaid first visual hull corresponding to said pixels.
 7. A non-transitorycomputer readable medium storing a program for constructing a visualhull from a number of object images in which an object and a backgroundare captured and a number of background images which only the backgroundis captured, the program, when executed by a computer, causing thecomputer to perform a method comprising: extracting a number of firstsilhouettes from the number of said object images and the number of saidbackground images by a background subtraction; constructing a firstvisual hull from the number of said first silhouettes by a shape fromsilhouette method; constructing a second visual hull by a process torepair missed parts or to remove unwanted regions in said first visualhull; extracting a number of second silhouettes from said second visualhull; and constructing a third visual hull from the number of saidsecond silhouettes by the shape from silhouette method, wherein saidsecond construct means includes a sub means for acquiring a number offirst slice images of said first visual hull from an x-axis, a y-axisand a z-axis direction; and a sub means for performing a filter processto the number of said first slice images and constructing said secondvisual hull according to a result of said filter process, and said submeans for constructing said second visual hull is a means forconstructing said second visual hull by performing a filter process tothe number of said first slice images, by searching for pixels thatbecame white by the filter process, wherein white represents the objectregion, and black represents background, and by repairing threedimensional coordinates of said first visual hull corresponding to saidpixels.